Image management method and data structure of metadata

ABSTRACT

The present technology relates to an image management method and a data structure of metadata that allow for management of an image captured by an artificial satellite. A management device that manages a captured image captured by a satellite adds metadata that includes at least information regarding a person related to the captured image to the captured image. The present technology can be applied to, for example, artificial satellites that perform satellite remote sensing by a formation flight.

TECHNICAL FIELD

The present technology relates to an image management method and a datastructure of metadata, and more particularly, to an image managementmethod and a data structure of metadata that allow for management of animage captured by an artificial satellite.

BACKGROUND ART

There is a technology called satellite remote sensing for observing asituation of a target area or a target object or detecting a change insituation from an image of a predetermined area on the earth captured byan artificial satellite (see, for example, Patent Documents 1 and 2).

CITATION LIST Patent Document

-   Patent Document 1: WO 2010/097921 A-   Patent Document 2: Japanese Patent Application Laid-Open No.    2004-15451

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

In recent years, research and development of low-cost small satellitesby private sectors have progressed. It is expected that images will becaptured by many artificial satellites in the future, and a system formanagement of an image captured by an artificial satellite is desired.

The present technology has been made in view of such a situation, andallows for management of an image captured by an artificial satellite.

Solutions to Problems

An image management method according to a first aspect of the presenttechnology includes adding, by a management device that manages acaptured image captured by a satellite, metadata that includes at leastinformation regarding a person related to the captured image, to thecaptured image.

In the first aspect of the present technology, the metadata thatincludes at least the information regarding the person related to thecaptured image captured by the satellite is added to the captured image.

A data structure of metadata according to a second aspect of the presenttechnology is a data structure of metadata of a captured image capturedby a satellite, in which the metadata includes at least informationregarding a person related to the captured image, and a managementdevice that manages the captured image is used for processing ofcollating the person related to the captured image.

In the second aspect of the present technology, the metadata thatincludes at least the information regarding the person related to thecaptured image captured by the satellite is used for the processing ofcollating the person related to the captured image.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration example of afirst embodiment of a satellite image processing system to which thepresent technology is applied.

FIG. 2 is a diagram illustrating a formation flight.

FIG. 3 is a diagram illustrating a formation flight.

FIG. 4 is a block diagram illustrating a configuration example of asatellite.

FIG. 5 is a block diagram illustrating a configuration example of asatellite cluster management device, a communication device, and animage analysis server.

FIG. 6 is a flowchart illustrating an imaging sequence focusing on onesatellite.

FIG. 7 is a detailed flowchart of imaging preparation processing in stepS33 in FIG. 6.

FIG. 8 is a diagram illustrating determination of a remaining batterylevel.

FIG. 9 is a flowchart of the satellite image processing system in whicha formation flight is performed.

FIG. 10 is a diagram illustrating information attached as metadata.

FIG. 11 is a diagram illustrating a configuration example of a secondembodiment of a satellite image processing system to which the presenttechnology is applied.

FIG. 12 is a block diagram illustrating a configuration example of atransmission device according to the second embodiment.

FIG. 13 is a flowchart illustrating a first event imaging sequence bythe satellite image processing system of the second embodiment.

FIG. 14 is a flowchart illustrating a second event imaging sequence bythe satellite image processing system of the second embodiment.

FIG. 15 is a flowchart illustrating a third event imaging sequence bythe satellite image processing system of the second embodiment.

FIG. 16 is a block diagram illustrating another configuration example ofthe transmission device according to the second embodiment.

FIG. 17 is a block diagram illustrating a configuration example of oneembodiment of a computer to which the present technology is applied.

MODE FOR CARRYING OUT THE INVENTION

Modes for carrying out the present technology (hereinafter referred toas “embodiments”) will be described below. Note that the descriptionwill be made in the order below.

1. Configuration example of satellite image processing system

2. Imaging sequence by single device

3. Imaging preparation processing

4. Flowchart of formation flight

5. Example of image processing

6. Details of metadata

7. Details of distribution management processing

8. Application example of formation flight

9. Second embodiment of satellite image processing system

10. First event imaging sequence of second embodiment

11. Second event imaging sequence of second embodiment

12. Third event imaging sequence of second embodiment

13. Another configuration example of transmission device

14. Application examples of satellite image processing system usingevent detection sensor

15. Configuration example of computer

1. Configuration Example of Satellite Image Processing System

FIG. 1 is a block diagram illustrating a configuration example of afirst embodiment of a satellite image processing system to which thepresent technology is applied.

A satellite image processing system 1 in FIG. 1 is a system that usescaptured images captured by a plurality of artificial satellites(hereinafter simply referred to as satellites) to perform satelliteremote sensing in which a situation of a target area or a target objecton the earth is observed or a change in situation is detected. In thepresent embodiment, a satellite is mounted with an imaging device, andhas at least a function of imaging the ground.

A satellite operation company has a satellite cluster management device11 that manages a plurality of satellites 21 and a plurality ofcommunication devices 13 that communicate with the satellites 21. Notethat the satellite cluster management device 11 and some of theplurality of communication devices 13 may be devices owned by a companyother than the satellite operation company. The satellite clustermanagement device 11 and the plurality of communication devices 13 areconnected via a predetermined network 12. The communication devices 13are disposed at ground stations (base stations on the ground) 15. Notethat FIG. 1 illustrates an example in which the number of thecommunication devices 13 is three, which are communication devices 13Ato 13C, but the number of the communication devices 13 is optional.

The satellite cluster management device 11 manages the plurality ofsatellites 21 owned by the satellite operation company. Specifically,the satellite cluster management device 11 acquires related informationfrom one or more information provision servers 41 of an externalinstitution as necessary, and determines an operation plan for theplurality of satellites 21 owned by the satellite cluster managementdevice 11 itself. Then, the satellite cluster management device 11instructs a predetermined satellite 21 to capture an image via acommunication device 13 in response to a request from a customer,thereby causing the predetermined satellite 21 to capture an image.Furthermore, the satellite cluster management device 11 acquires andstores the captured image transmitted from the satellite 21 via thecommunication device 13. The acquired captured image is subjected topredetermined image processing as necessary, and provided (transmitted)to the customer. Alternatively, the acquired captured image is provided(transmitted) to an image analysis server 42 of an image analysiscompany, subjected to predetermined image processing, and then providedto the customer.

The information provision server 41 installed in the externalinstitution supplies predetermined related information to the satellitecluster management device 11 via a predetermined network in response toa request from the satellite cluster management device 11 or on aperiodic basis. The related information provided from the informationprovision server 41 includes, for example, the following. For example,orbit information of a satellite described in a Two Line Elements (TLE)format can be acquired as related information from North AmericanAerospace Defense Command (NORAD) as an external institution.Furthermore, for example, it is possible to acquire weather informationsuch as weather and cloud cover at a predetermined point on the earthfrom a weather information providing company as an external institution.

The image analysis server 42 performs predetermined image processing onthe image captured by the satellite 21 supplied from the satellitecluster management device 11 via a predetermined network. The processedimage is provided to a customer of the image analysis company, orsupplied to the satellite cluster management device 11 of the satelliteoperation company. For example, the image analysis server 42 performsmetadata generation processing for adding predetermined metadata to animage captured by the satellite 21, correction processing such asdistortion correction of the captured image, image compositionprocessing such as color composition processing, and the like. The imageprocessing of the captured image may be performed by a satelliteoperation company, and in this case, the satellite operation company andthe image analysis company are the same. Furthermore, the satellitecluster management device 11 and the image analysis server 42 may beconstituted by one device.

In accordance with the control of the satellite cluster managementdevice 11, the communication device 13 communicates with a predeterminedsatellite 21 designated by the satellite cluster management device 11via an antenna 14. For example, the communication device 13 transmits,to a predetermined satellite 21, an imaging instruction for imaging apredetermined region on the ground at a predetermined time and position.Furthermore, the communication device 13 receives the captured imagetransmitted from the satellite 21, and supplies the captured image tothe satellite cluster management device 11 via the network 12.Transmission from the communication device 13 of the ground station 15to the satellite 21 is also referred to as uplink, and transmission fromthe satellite 21 to the communication device 13 is also referred to asdownlink. The communication device 13 can perform communication directlywith the satellite 21, and can also perform communication via a relaysatellite 22. As the relay satellite 22, for example, a geostationarysatellite is used.

The network 12 or a network between the information provision server 41or the image analysis server 42 and the satellite cluster managementdevice 11 is an optional communication network, and may be a wiredcommunication network, may be a wireless communication network, or maybe constituted by both of them. Furthermore, the network 12 and thenetwork between the information provision server 41 or the imageanalysis server 42 and the satellite cluster management device 11 may beconstituted by one communication network, or may be constituted by aplurality of communication networks. These networks can be acommunication network or a communication path compliant with an optionalcommunication standard, for example, the Internet, a public telephoneline network, a wide area communication network for a wireless movingobject such as a so-called 4G line or 5G line, a wide area network(WAN), a local area network (LAN), a wireless communication network forcommunication that meets a Bluetooth (registered trademark) standard, acommunication path for short-range wireless communication such as nearfield communication (NFC), a communication path for infraredcommunication, or a communication network for wired communication thatmeets a standard such as high-definition multimedia interface (HDMI(registered trademark)) or universal serial bus (USB).

A plurality of the satellites 21 constitutes a satellite cluster 31. InFIG. 1, a satellite 21A and a satellite 21B constitute a first satellitecluster 31A, and a satellite 21C and a satellite 21D constitute a secondsatellite cluster 31B. Note that the example in FIG. 1 illustrates, forthe sake of simplicity, an example in which one satellite cluster 31 isconstituted by two satellites 21, but the number of satellites 21constituting one satellite cluster 31 is not limited to two.

In a case where the communication device 13 communicates with thesatellites 21 constituting the satellite cluster 31, there are a methodin which communication is performed individually with the satellites 21as in the first satellite cluster 31A in FIG. 1, and a method in whichonly one satellite 21C (hereinafter also referred to as therepresentative satellite 21C) representing the satellite cluster 31communicates with the communication device 13 and the other satellite21D indirectly communicates with the communication device 13 byinter-satellite communication with the representative satellite 21C asin the second satellite cluster 31B. By which method communication withthe ground station 15 (the communication device 13 thereof) is to beperformed may be determined in advance by the satellite cluster 31, ormay be appropriately selected in accordance with contents ofcommunication.

In the satellite image processing system 1 configured as describedabove, the plurality of satellites 21 constituting one satellite cluster31 may be operated by an operation method called formation flight.

The formation flight is an operation method in which, as illustrated inFIG. 2, a plurality of satellites 21 constituting one satellite cluster31 flies while maintaining a relative positional relationship in anarrow range of about several hundred meters to several kilometers, andthe plurality of satellites 21 operates in a coordinated manner, so thata service that cannot be provided by a single satellite can be provided.In FIG. 2, three satellites 21X to 21Z constitute one satellite cluster31, and each of the satellites 21X to 21Z communicates with the groundstation 15. In the uplink, a cluster ID (satellite cluster ID), which isan identifier for identifying the satellite cluster 31, and anindividual ID (satellite ID), which is an identifier for identifyingeach satellite 21 constituting the satellite cluster 31, are designated,so that a command or data is transmitted to a desired satellite 21.

The formation flight allows functions to be assigned to a plurality ofsatellites 21 instead of a single satellite, and therefore has anadvantage that the satellites 21 can be downsized. For example, as forimaging functions, even in a case where a performance (e.g., theresolution) of the imaging device mounted on each satellite 21 islowered, high resolution can be achieved by image composition or thelike of captured images captured by the plurality of satellites 21.

For example, as illustrated in A of FIG. 3, two satellites 21E and 21Fcan simultaneously image one region 52 (simultaneous imaging) fromdifferent imaging points (satellite positions). A result of imaging thesame ground surface from different imaging points can be used forgeneration of a digital elevation model (DEM) indicating a heightnecessary for three-dimensional measurement. Furthermore, a parallaximage is obtained from the captured images from the two satellites 21Eand 21F, and three-dimensional measurement can be performed.

Furthermore, as illustrated in B of FIG. 3, a plurality of satellites21E and 21F can image one region 52 with a time difference (differentialimaging) at the same imaging point and imaging angle. For example, in acase where the satellites 21 are moving at a speed of 7 km per secondand flying in formation with a distance of 100 m between the satellites21, imaging can be performed every 1.4×10⁻² seconds. As described above,a formation flight allows for imaging at a short time interval, andthus, for example, it is possible to extract a change (displacement) inan object on the earth such as a passenger car on a road or a buoy onthe sea, and to measure the speed of a moving object.

There is a constellation as a system for operating a plurality ofsatellites 21, but the constellation is “a system that mainly deploys auniform global service by putting a large number of satellites into asingle or a plurality of orbital planes”, which is a concept differentfrom that of the formation flight.

FIG. 4 is a block diagram illustrating a configuration example of asatellite 21.

The satellite 21 includes a management unit 101, a bus 102, an imagingcontrol unit 103, a heat control unit 104, an attitude control systemcontrol unit 105, an orbit control system control unit 106, a propulsionsystem control unit 107, a sensor control unit 108, a power supplycontrol unit 109, and a communication control unit 110. Furthermore, thesatellite 21 also includes an imaging device 111, a cooling device 112,an attitude control device 113, a propulsion device 114, a sensor group115, a battery 116, a solar panel 117, and a communication device 118.The management unit 101 and control units for devices are connected viathe bus 102, the control units including the imaging control unit 103,the heat control unit 104, the attitude control system control unit 105,the orbit control system control unit 106, the propulsion system controlunit 107, the sensor control unit 108, the power supply control unit109, and the communication control unit 110.

The management unit 101 acquires states of the devices from thecorresponding control units for the devices via the bus 102, and outputsan operation command to the control units for the devices, therebycontrolling an operation of the entire satellite 21.

The imaging control unit 103 controls an operation of the imaging device111 in accordance with the operation command from the management unit101. The imaging device 111 is constituted by, for example, a cameramodule including an image sensor, and images a target object. In a casewhere the satellite 21 is a synthetic aperture radar (SAR) satellite,the imaging device 111 is constituted by a radar device.

The heat control unit 104 acquires a sensor value of a temperaturesensor included in the sensor group 115, monitors a temperature changein the satellite 21, and performs control to cause the entire satellite21 to be within a prescribed temperature range. Basically, thetemperature change is controlled by a structure or a characteristic of amaterial, but dynamic cooling using the cooling device 112 may beperformed as necessary. The cooling device 112 performs cooling by usinga cryogen such as liquid helium, for example.

The attitude control system control unit 105 controls the attitudecontrol device 113 in accordance with an operation command from themanagement unit 101 to perform control to turn the satellite 21 in anintended direction. For example, the attitude control system controlunit 105 performs control to turn the antenna 14 toward the groundstation 15, turn the solar panel 117 toward the sun, or turn anobservation sensor of the imaging device 111 or the like in thedirection of an observation target. The attitude control device 113 isconstituted by, for example, a wheel such as a three-axis gyroscope or acontrol moment gyroscope, a magnetic torquer, and the like. The attitudecontrol system control unit 105 may use not only the attitude controldevice 113 but also the propulsion device 114 for attitude control. Whenperforming attitude control, the attitude control system control unit105 acquires sensor values of various sensors of the sensor group 115 asnecessary. Examples of the sensors used for the attitude control includea sun sensor, an earth sensor, a star sensor, a magnetic sensor, and agyroscope.

The orbit control system control unit 106 performs control related tomaintaining an orbit altitude and changing the orbit. The orbit controlsystem control unit 106 performs control in cooperation with thepropulsion system control unit 107 and the propulsion device 114.

The propulsion system control unit 107 controls the propulsion device114 in accordance with an operation command from the management unit101. The propulsion device 114 is constituted by, for example, a solidmotor, an ion engine, or an apogee engine. The propulsion system controlunit 107 acquires sensor values of the various sensors of the sensorgroup 115 and operates the propulsion device 114 in cooperation with theattitude control device 113 as necessary, thereby performing attitudecontrol and attitude control for the satellite 21. In a case where thesatellite 21 is a small satellite, a chemical propulsion thruster or thelike may not be mounted for the purpose of attitude control.

The sensor control unit 108 controls the various sensors included in thesensor group 115, and supplies sensor values to the management unit 101or to another control unit. The various sensors are sensors formonitoring the state in the satellite 21, and include, for example, aGPS receiver, a star tracker (attitude sensor), an acceleration sensor,a gyroscope sensor, a magnetic sensor, a temperature sensor, a sunsensor, an earth sensor, and a star sensor.

The power supply control unit 109 controls the battery 116 and the solarpanel 117. Power generated by the solar panel 117 is stored in thebattery 116 under the control of the power supply control unit 109. Thepower in the battery 116 may be directly distributed to the devices inthe satellite 21, or may be distributed via the bus 102.

The communication control unit 110 controls the communication device 118in accordance with an operation command from the management unit 101.The communication device 118 has an antenna, and communicates with thecommunication device 13 of the ground station 15 in accordance with thecontrol of the communication control unit 110. Furthermore, thecommunication device 118 can also communicate with another satellite 21constituting the same satellite cluster 31 and with the relay satellite22. Furthermore, the communication control unit 110 and thecommunication device 118 may have separate systems, one for transmissionand reception of commands and telemetry, which are small in data amount,and one for mission-related data (imaging data and the like), which islarge in data amount.

The control units from the imaging control unit 103 to the communicationcontrol unit 110 may be further divided into two or more, any two ormore of the control units may be integrated, or the control units may beintegrated with the management unit 101. Computational resource such asa central processing unit (CPU) and a memory are basically mounted onthe management unit 101, but may also be mounted on the control units.The control units may be implemented in a common hardware module.

The imaging devices 111 of a plurality of the satellites 21 constitutingone satellite cluster 31 may have the same performance, or may havedifferent performances.

For example, in a case where imaging devices 111 of the same modelnumber are adopted as the imaging devices 111 mounted on the satellites21 so that the satellites 21 have the same performance, there are thefollowing advantages. For example, images of the same performance can beacquired with a short time difference, and the difference can be easilydetected. Furthermore, it is possible to generate a highly accurate(high-resolution) image by, for example, compositing images captured inaccordance with an assignment. Furthermore, this allows for aredundancy, which makes a malfunction in one device tolerable.

On the other hand, in a case where the imaging devices 111 mounted onthe satellites 21 have different performances, it is possible to assigndifferent roles in imaging, for example, one for high-sensitivitymonochrome imaging and one for low-sensitivity color imaging. Note thatthe different performances includes not only a case where the mountedhardware configurations are different but also a case where the mountedhardware configurations are the same but the performances are differentdue to a difference in control. For example, an example is assumed inwhich, in a case where the image sensors are of the same model number,one satellite 21 acquires a high-sensitivity low-resolution image with afaster shutter speed, and another satellite 21 acquires alow-sensitivity high-resolution image in an opposite way.

As an assignment example in a case where the imaging devices 111 of theplurality of satellites 21 have different performances, there may be,for example, control to make any one of sensitivity/shutter speed,resolution, monochrome/color/polarization, or band (wavelength region),or a combination thereof, different. Furthermore, the plurality ofsatellites 21 may be different in battery performance or communicationperformance.

FIG. 5 is a block diagram illustrating a configuration example of thesatellite cluster management device 11, the communication device 13, andthe image analysis server 42.

The satellite cluster management device 11 includes a control unit 211,a communication unit 212, an operation unit 213, and a display unit 214.

The control unit 211 manages the plurality of satellites 21 owned by thesatellite operation company by executing a satellite managementapplication program stored in a storage unit (not illustrated). Forexample, the control unit 211 determines an operation plan for theplurality of satellites 21 by using related information acquired fromthe information provision server 41 as necessary, and instructs thesatellites 21 to control the attitude or capture an image via thecommunication device 13. Furthermore, the control unit 211 performs, forexample, processing of generating metadata of a captured imagetransmitted from a satellite 21 via the communication device 13 andadding the metadata to the captured image.

In accordance with an instruction from the control unit 211, thecommunication unit 212 performs a predetermined communication with thecommunication device 13 via the network 12, and also performs apredetermined communication with the image analysis server 42.

The operation unit 213 is constituted by, for example, a keyboard, amouse, and a touch panel, receives an input of a command or data basedon a user (operator) operation, and supplies the command or data to thecontrol unit 211.

The display unit 214 is constituted by, for example, a liquid crystaldisplay (LCD) or an organic electro luminescence (EL) display, anddisplays a screen of the satellite management application program, ordisplays a captured image captured by the satellite 21, a processedimage obtained by performing predetermined image processing on thecaptured image, or the like.

The communication device 13 includes a satellite communication unit 221,a control unit 222, and a communication unit 223.

The satellite communication unit 221 communicates with the satellites 21of a target satellite cluster 31 via the antenna 14 on the basis of thecontrol of the control unit 222.

The control unit 222 causes the satellite communication unit 221 tocommunicate with a satellite 21 in accordance with the control from thesatellite cluster management device 11. Furthermore, the control unit222 transmits data such as a captured image acquired from the satellite21 to the satellite cluster management device 11 via the communicationunit 223.

The communication unit 223 performs a predetermined communication withthe satellite cluster management device 11 on the basis of the controlof the control unit 222.

The image analysis server 42 includes a control unit 231, acommunication unit 232, an operation unit 233, and a display unit 234.

By executing an image analysis application program stored in a storageunit (not illustrated), the control unit 231 performs, on a capturedimage supplied from the satellite cluster management device 11,predetermined image processing such as metadata generation processingfor adding predetermined metadata to the captured image, correctionprocessing for distortion correction or the like of the captured image,or image composition processing such as color composition processing.

The communication unit 232 performs a predetermined communication withthe satellite cluster management device 11 or another device inaccordance with the control from the control unit 231. For example, thecommunication unit 232 receives a captured image captured by thesatellite 21 from the satellite cluster management device 11 andsupplies the captured image to the control unit 231, or transmits aprocessed image after image processing to the satellite clustermanagement device 11.

The operation unit 233 is constituted by, for example, a keyboard, amouse, and a touch panel, receives an input of a command or data basedon a user (operator) operation, and supplies the command or data to thecontrol unit 231.

The display unit 214 is constituted by, for example, an LCD or anorganic EL display, and displays a screen of the image analysisapplication program or displays an image before or after imageprocessing.

The satellites 21 and other devices constituting the satellite imageprocessing system 1 are configured as described above.

Note that the satellite cluster management device 11 selects an optimalcommunication device 13 from among the plurality of communicationdevices 13 in accordance with the orbit of a satellite 21 with whichcommunication is to be performed, and causes the selected communicationdevice 13 to transmit a predetermined command such as an imaginginstruction or receive data such as a captured image via thecommunication device 13. Since the satellite cluster management device11 performs a predetermined communication integrally with thecommunication device 13 optionally selected in accordance with thetarget satellite 21, the satellite cluster management device 11 and thecommunication device 13 will be collectively referred to as a managementsystem in the following description.

2. Imaging Sequence by Single Device

Next, an imaging sequence focusing on one predetermined satellite 21 ofthe satellite cluster 31 that performs a formation flight will bedescribed with reference to a flowchart in FIG. 6.

First, in step S11, a management system determines requirements forimaging by the satellite 21 on the basis of a request from a customer.

Specifically, the management system determines, as the imagingrequirements, an imaging date and time, an imaging point, anenvironmental condition for imaging, a camera setting value, and thelike. The environmental condition for imaging includes, for example, aweather condition such as cloud cover at the imaging date and time, andthe camera setting value includes, for example, the resolution(resolving power), zoom, shutter speed, sensitivity, and aperture.

In step S12, the management system determines the satellite 21 and theground station 15 (the communication device 13 thereof) that meet theimaging requirements.

Specifically, the management system selects the satellite 21 that meetsthe determined imaging requirements. For example, the satellite 21 isdetermined on the basis of determinations on whether the satellite 21passes over an imaging target position at the determined imaging dateand time, whether the imaging target position is within the range ofobservation width of the satellite 21, whether the imaging device 111mounted on the satellite 21 satisfies requirements such as the resolvingpower and the determined camera setting value, and the like. Then, theground station 15 suitable for communicating with the selected satellite21 is determined.

Furthermore, the management system can select the satellite 21 inconsideration of an expected remaining battery level of the satellite 21at the imaging date and time, a power consumption for imaging, and thelike. For example, in a case where the selected satellite 21 is plannedto perform another imaging immediately before the imaging date and time,power is consumed by the imaging, attitude control, data communication,heat control, and the like associated with the imaging, and it isassumed that the next imaging may not be able to be performed. Thus, adegree of priority of the satellite 21 is set in accordance with theexpected remaining battery level and the power consumption for theimaging, and the satellite 21 is selected.

In step S13, the management system directs the antenna 14 of theselected ground station 15 toward an assumed orbit. The satellitecluster management device 11 transmits orbit information of the selectedsatellite 21 to the communication device 13, and the communicationdevice 13 directs the antenna 14 toward the assumed orbit.

In step S14, the management system transmits (uplinks) an imaginginstruction to the selected satellite 21. That is, the satellite clustermanagement device 11 transmits a command for transmitting an imaginginstruction to the communication device 13 of the selected groundstation 15, and the communication device 13 that has received thecommand transmits the imaging instruction to the selected satellite 21via the antenna 14. The imaging instruction includes an imaging date andtime, an imaging point, a camera setting value, and the like.

In step S31, the satellite 21 receives the imaging instruction from theground station 15, and in step S32, transmits a reception completion tothe ground station 15.

In step S15, the management system receives the reception completionfrom the satellite 21, and stops transmitting the imaging instruction.The transmission of the imaging instruction from the ground station 15is repeated until the satellite 21 returns the reception completion.

In step S33, the satellite 21 performs imaging preparation processingbased on the received imaging instruction. For example, the satellite 21controls the attitude of the satellite 21 or an orientation of theimaging device 111 (pointing) such that the imaging device 111 turnstoward the imaging target position as necessary. Furthermore, forexample, the imaging control unit 103 sets zoom, shutter speed,sensitivity, aperture, and the like of the image sensor. Moreover, thepower supply control unit 109 performs charging in advance so thatsufficient power is obtained at the imaging date and time.

At the imaging date and time designated by the imaging instruction, thesatellite 21 images the imaging target position in step S34.

In step S35, the satellite 21 generates metadata, which is informationto be associated with a captured image obtained as a result of theimaging, and adds the metadata to the captured image. Although detailsof the metadata will be described later, for example, information suchas a cluster ID for identifying the satellite cluster 31, an individualID for identifying each satellite 21, an imaging target position(subject position), and an imaging time can be generated as themetadata.

In step S36, the satellite 21 transmits (downlinks) the captured imageto which the metadata has been added to the ground station 15. Thedownlink may be performed immediately after generation of the capturedimage and the metadata, or may be performed at the time of arrivalwithin a predetermined range in a predetermined ground station 15.Furthermore, the captured image may be transmitted via the relaysatellite 22.

In step S16, the management system receives the captured image from thesatellite 21. Specifically, the communication device 13 receives thecaptured image via the antenna 14, and supplies the captured image tothe satellite cluster management device 11.

In step S17, the satellite cluster management device 11 analyzes themetadata of the captured image. At this time, the satellite clustermanagement device 11 may newly generate metadata on the basis of aresult of the analysis, and add this metadata. For example, thesatellite cluster management device 11 calculates a satellite positionat the time of imaging on the basis of the cluster ID and the individualID of the captured image and the orbit information of the satellite 21,and adds the satellite position as metadata.

In step S18, the satellite cluster management device 11 performspredetermined image processing on the captured image captured by thesatellite 21. The satellite cluster management device 11 performs, forexample, correction processing such as distortion correction and imagecomposition processing such as color composition processing. Details ofthe image processing will be described later.

In step S19, the satellite cluster management device 11 executesdistribution management processing on the captured image and theprocessed image, and stores the captured image and the processed imagein a predetermined storage unit. Details of the distribution managementprocessing will also be described later.

Thus, a series of sequences for imaging by one satellite 21 ends. Notethat the image processing by the image analysis server 42 can beappropriately performed as necessary, and can be performed in a sharedmanner with the image processing by the satellite cluster managementdevice 11 or performed instead of being performed by the satellitecluster management device 11. In a similar manner, the distributionmanagement processing may be performed by the image analysis server 42.

Note that, in the above-described example, the metadata is added to thecaptured image and transmitted, but the metadata may be transmitted as astream different from the captured image. At this time, only themetadata may be transmitted prior to the captured image.

3. Imaging Preparation Processing

Incidentally, resources are limited particularly in a small satellite21, and thus, it is necessary to pay particular attention to theremaining battery level, and it is important to control imaging inaccordance with this.

FIG. 7 is a detailed flowchart of the imaging preparation processing instep S33 in FIG. 6. Here, it is assumed that the imaging instructionreceived in step S31 before step S33 is an instruction to capture animage at imaging time t1.

In the imaging preparation processing, first, in step S51, themanagement unit 101 of the satellite 21 estimates the remaining batterylevel at the imaging time t1. Specifically, the management unit 101estimates the remaining battery level at the imaging time t1 from (anestimated value of) a charge capacity accumulated by solar powergeneration by the imaging time t1 with respect to the current remainingbattery level.

In step S52, the management unit 101 determines whether the remainingbattery level is sufficient on the basis of the estimated remainingbattery level.

Specifically, the management unit 101 determines whether the estimatedremaining battery level is sufficient from factors of power consumptionrelated to imaging and factors of power consumption that are not forimaging. The factors of power consumption related to imaging includeimaging processing by the imaging device 111, attitude control(pointing) for the satellite 21, and heat control associated therewith.The imaging processing by the imaging device 111 takes intoconsideration how many images are to be captured with what degree ofaccuracy (resolving power, shutter speed, necessity of zoom, and thelike) at the imaging time t1. The attitude control for the satellite 21includes a change in the attitude of the satellite itself and a changein the attitude of the antenna. Furthermore, in a case where the cameramodule itself as the imaging device 111 can change the attitude in thedirection of imaging, a change in the attitude of the camera module isalso included in the attitude control for the satellite 21. The factorsof power consumption that are not for imaging include communication(uplink and downlink) performed by the imaging time t1.

For example, as illustrated in A of FIG. 8, on the premise that a chargecapacity of 70% of a full charge capacity of the battery 116 is to bemaintained at all times, assuming that the current remaining batterylevel is 90%, the charge capacity by the time t1 is 5%, the powerconsumption by the imaging processing at the time t1 is 3%, the powerconsumption by the attitude control is 10%, and the power consumption bythe communication performed until the imaging time t1 is 2%,90%+5%−3%−10%−2%=80% is obtained. The charge capacity of 70% is ensuredeven after the imaging at the time t1, and the satellite 21 isdetermined to have a sufficient remaining battery level.

Note that the management unit 101 may determine whether the remainingbattery level is sufficient on the basis of the remaining battery levelto be kept after the imaging time t1 also in consideration of imagingperformed at a timing subsequent to the imaging time t1.

For example, as illustrated in B of FIG. 8, assuming that imaging isscheduled at time t2 subsequent to the imaging time t1, the chargecapacity from the time t1 to the time t2 is 2%, the power consumption bythe imaging processing at the time t2 is 3%, the power consumption bythe attitude control is 10%, and the power consumption by thecommunication performed until the imaging time t2 is 2%, the remainingbattery level of 83% is required after the imaging at the time t1. Thus,it is determined that the estimated remaining battery level 80% at theimaging time t1 is not a sufficient remaining battery level.

Note that the above-described example mainly describes power consumptionrelated to imaging, but power consumption for others such as powerconsumption by heat control associated with attitude control, periodiccommunication, and the like is also taken into consideration.

As described above, whether or not the remaining battery level issufficient is determined. If it is determined in step S52 in FIG. 7 thatthe remaining battery level is not sufficient, the processing proceedsto step S53, and the satellite 21 determines whether an assumed downlinktiming before the imaging time t1 can be changed. By changing thedownlink timing, it is possible to save the amount of power required forthe downlink.

If it is determined in step S53 that the downlink timing cannot bechanged, the processing in step S53 is skipped, and the processingproceeds to step S55.

On the other hand, if it is determined in step S53 that the downlinktiming can be changed, the processing proceeds to step S54. Themanagement unit 101 changes the downlink timing, and determines whetherthe remaining battery level is sufficient after the change. If it isdetermined also in step S54 that the remaining battery level is notsufficient, the processing proceeds to step S55. On the other hand, ifit is determined in step S54 that the remaining battery level issufficient, the processing proceeds to step S57.

In step S55, the management unit 101 changes an accuracy of the attitudecontrol. In the attitude control, for example, two types, that is, awheel and an ion engine, are used to repeat applying a moment toward atarget attitude and then applying a reverse moment when the attitude haspassed through the target attitude. When a swing speed has become equalto or less than a certain value, it is determined that the attitude hasbeen changed to the target attitude. To change the accuracy of theattitude control, the management unit 101 changes, for example, therange of the swing speed used to determine that the target attitude hasbeen obtained. An electricity consumption can be saved by changing therange of the swing speed to increase and reducing the control amount ofthe attitude control.

In step S56, the management unit 101 changes an imaging condition inaccordance with the accuracy of the attitude control. When the range ofthe swing speed increases, the attitude of the satellite 21 is notstabilized and wobble occurs, which may cause subject blurring.Furthermore, the pointing is insufficient, and it is conceivable thatsufficient zoom cannot be achieved. Thus, the management unit 101changes an imaging condition to compensate for the adverse effectscaused by the reduction in the control amount of the attitude control.

For example, the management unit 101 changes an imaging condition asfollows.

The management unit 101 increases the shutter speed of the image sensorto cope with subject blurring. Furthermore, moreover, since the capturedimage becomes dark when the shutter speed is increased, the managementunit 101 may perform control to increase the sensitivity (gain).

Furthermore, for example, the management unit 101 can reduce theresolving power (resolution) of the captured image for the purpose ofimproving the sensitivity per unit pixel. With this arrangement, theshutter speed can be improved, an influence of the decrease in theaccuracy of the attitude control is reduced, and the amount of data atthe time of downlink can be reduced. Furthermore, the management unit101 selects a setting value for not using optical zoom. With thisarrangement, a tolerance for image blurring (wobble) can be increased.

Furthermore, in a case where the camera module has a mechanical blurcorrection mechanism (spatial blurring correction), the mechanical blurcorrection mechanism may be performed instead of reducing the accuracyof the attitude control.

Furthermore, instead of reducing the resolving power (resolution) of thecaptured image, the management unit 101 may configure an imaging settingfor continuously capturing a plurality of images. A high-resolutioncaptured image composited and generated from the continuously capturedimages is generated and transmitted (downlinked) to the ground station15, so that a decrease in resolving power (resolution) of the capturedimage can be compensated. Note that generation of the high-resolutionimage by image composition may be performed by the satellite clustermanagement device 11 or the image analysis server 42 after downlink. Thesatellite cluster management device 11 or the image analysis server 42can also perform composition by using a past captured image such as abase image or a captured image captured by another satellite 21.

After step S56, or if it is determined in step S52 or step S54 that theremaining battery level is sufficient, the processing proceeds to stepS57.

In step S57, the management unit 101 controls the attitude of thesatellite 21 or the imaging device 111 (performs pointing) in accordancewith the setting of the attitude control determined in the processing instep S55.

In step S58, the management unit 101 sets the imaging conditiondetermined in the processing in step S56.

Thus, the imaging preparation processing in step S33 in FIG. 6 ends. Atthe imaging date and time designated by the imaging instruction, theprocessing in step S34 in FIG. 6, that is, imaging of the imaging targetposition is performed.

According to the imaging preparation processing, in a case where theremaining battery level is low, the accuracy of stabilization in theattitude control that greatly affects the electricity consumption islowered, and an imaging condition or the image processing in thesubsequent stage is changed. This makes it possible to secure a qualityof the captured image while suppressing a battery consumption.

4. Flowchart of Formation Flight

Next, a formation flight executed by the plurality of satellites 21constituting one satellite cluster 31 will be described.

FIG. 9 is a flowchart of the satellite image processing system 1 inwhich one satellite cluster 31 performs a formation flight.

First, relative position checking processing in steps S101, S121, S122,and S102 is performed between the management system and the satellites21 of the satellite cluster 31 that performs a formation flight. Thatis, in step S101, the management system inquires, about the relativeposition, of the satellites 21 of the satellite cluster 31 performingthe formation flight. In step S121, the satellites 21 constituting thesatellite cluster 31 perform processing of checking the relativeposition in response to the inquiry from the management system. Then, instep S122, the satellites 21 transmit the relative position. In stepS102, the management system receives the relative position from eachsatellite 21. Here, the relative position indicates an arrangementsequence of the satellites 21 constituting the satellite cluster 31 anddistances between the satellites. The arrangement sequence of thesatellites 21 is, for example, an order in which the top (No. 1) is inthe traveling direction of the satellites 21. The relative positionchecking processing may be performed every time an image is captured, ormay be performed on a periodic basis, for example, once a day or once aweek.

The management system has orbit information of the satellite cluster 31acquired from NORAD as an external institution, but it may not bepossible to determine orbit information of each satellite 21constituting the satellite cluster 31. Alternatively, even in a casewhere individual orbit information can be determined by observation fromthe ground, it may not be possible to determine the order of airframes.In the formation flight, there is a case where the satellites 21 aredisposed in a range in which the orbit information cannot beindividually allocated, and it is not possible to determine the positiona certain satellite is placed from the top in the satellite cluster 31.It is therefore necessary to measure the relative positionalrelationship.

Methods for controlling a relative position are roughly classified intotwo types: an open-loop method and a closed-loop method.

The open-loop method is a method in which there is no communicationbetween satellites constituting the satellite cluster 31 and therelative position is controlled by an instruction from the ground side.An error is likely to occur in the distances between the satellites.

On the other hand, the closed-loop method is a method of controlling therelative position by performing communication between satellitesconstituting the satellite cluster 31. The closed-loop method has higheraccuracy in relative position than the open-loop method. The closed-loopmethod includes a centralized type and a decentralized type. In thecentralized type, there are a mode in which a satellite 21 serves as aleader and other satellites 21 follow the leader satellite, and a modein which the leader satellite gives an instruction to the othersatellites 21. The decentralized type is a mode in which each of thesatellites 21 constituting the satellite cluster 31 autonomouslycommunicates with other surrounding satellites 21 and controls its ownposition.

In the processing of checking the relative position in step S121, in acase of the open-loop method, for example, there is a method in whichthe satellites 21 simultaneously image a predetermined point on theground, and the satellite cluster management device 11 on the groundside checks the arrangement sequence of the satellites 21 on the basisof the captured images and information regarding the attitude (pointingangle) of each satellite 21. Furthermore, for example, there is a methodin which the satellites 21 perform communication simultaneously with apredetermined point on the ground, and the communication device 13 onthe ground side checks the arrangement sequence from the radio waves atthat time. The communication for checking the arrangement sequence maybe a downlink of a predetermined captured image, a signal forcalibration, or the like. On the other hand, in a case of theclosed-loop method, the satellites 21 execute processing of measuringthe relative position, and the measurement result is downlinked. Thesatellites 21 measure the relative position by a method such as a methodof measuring the position (direction) by communication betweensatellites, or a method of radiating laser from the satellites 21 andmeasuring the distances on the basis of its reflected light.

In the closed-loop method and the open-loop method, only the arrangementsequence of the satellites 21 may be detected, and the distances betweenthe satellites may be calculated by observation from the ground.

In step S103, the management system calculates imaging conditions ofeach satellite 21 on the basis of the relative position of eachsatellite 21. The imaging conditions here include, in addition to asetting value of the image sensor, attitude control for the satellite 21at the time of imaging, imaging timing, and the like. For example, in acase where three-dimensional measurement of the ground is performed, animaging condition for causing the satellites 21 to be in attitudes inwhich the imaging target positions are the same is calculated with aninter-satellite distance as a baseline length. In a case where imagesare captured with a time difference (differential imaging) by theplurality of satellites 21 constituting the satellite cluster 31, thetimings (imaging positions) at which a preceding satellite 21 and asubsequent satellite 21 capture images and the attitude at the time ofimaging are calculated. The timings at which the satellites 21 captureimages are calculated on the basis of the inter-satellite distance.

In step S104, the management system transmits an imaging instruction toeach satellite 21 on the basis of the calculated imaging conditions. Theimaging instruction is transmitted to all the satellites 21 of thesatellite cluster 31 (multicast), and each satellite 21 can select aninstruction addressed to itself by the individual ID as destinationinformation included in the imaging instruction.

In step S123, the satellites 21 receive an imaging instruction from theground station 15, perform imaging preparation processing in step S124,and capture images in step S125. Moreover, the satellites 21 generateand add metadata to the captured images in step S126, and transmit(downlink) the captured images to which the metadata has been added tothe ground station 15 in step S127.

The processing in steps S123 to S127 is basically similar to theprocessing in steps S31 to S36 performed by each satellite 21 describedwith reference to FIG. 6. Note that, when the captured images aretransmitted in step S127, each satellite 21 may individually transmit animage captured by the satellite 21, or the captured images may becollected in the leader satellite by inter-satellite communication andthen collectively transmitted by the leader satellite.

In step S105, the management system receives the captured image fromeach satellite 21, and analyzes the metadata of the captured image instep S106. Moreover, in step S107, the management system performspredetermined image processing on the captured image. In step S108, themanagement system executes distribution management processing on thecaptured image and the processed image, and stores the captured imageand the processed image in a predetermined storage unit.

The processing in steps S105 to S108 is basically similar to theprocessing in steps S16 to S19 performed by the management systemdescribed with reference to FIG. 6. However, in the image processing instep S107, not only image processing on a captured image obtained by onesatellite 21 but also image processing using a plurality of capturedimages captured in cooperation by the plurality of satellites 21 of thesatellite cluster 31 can be performed.

5. Example of Image Processing

A processing example of image processing executed by the satellitecluster management device 11 or the image analysis server 42 in step S18in FIG. 6 or step S107 in FIG. 9 will be described.

The satellite cluster management device 11 or the image analysis server42 can perform the following image processing on one captured imagecaptured by each satellite 21.

(1) Generation of Metadata

Metadata can be generated on the basis of information transmitted fromthe satellite 21 or information regarding the satellite 21 that hascaptured the image. For example, information regarding latitude andlongitude of an imaging target position, and information regardingattitude control and acceleration at the time of imaging by thesatellite 21 can be generated as metadata.

(2) Correction Processing of Captured Image

It is possible to perform correction processing such as radiometriccorrection related to a sensitivity characteristic, geometric correctionof an orbit position, an attitude error, or the like of the satellite21, ortho-correction for correcting geometric distortion caused by aheight difference of terrain, and map projection for projecting an imageon a map projection surface.

(3) Color Composition Processing

It is possible to perform color composition processing such aspan-sharpening processing, true color composition processing, falsecolor composition processing, natural color composition processing, SARimage composition processing, and processing of adding color to acaptured image for each band.

(4) Other Image Compositions

It is also possible to perform composition using a captured imagecaptured by the satellite 21 itself in the past, a captured imagecaptured by another satellite 21, or a base image of some kind,composition using captured images captured in different bands,composition using map information, and the like.

(5) Extraction of Information

It is possible to calculate vegetation detection information such as anormalized difference vegetation index (NDVI) and water detectioninformation such as a normalized difference water index (NDWI) bydifferent bands such as red (R) and infrared (IR). It is possible toperform highlight processing of a specific subject such as a vehicle, amoving object, or a group of fish, extraction of information regarding aspecific band or a change from the previous imaging, and the like.

In particular, in a case of using a plurality of captured imagescaptured by the plurality of satellites 21 that performs a formationflight, the satellite cluster management device 11 or the image analysisserver 42 can more effectively perform the following image processing.

(1) Higher Resolution or High Quality Processing

By superimposing a plurality of captured images, it is possible togenerate a captured image with improved resolving power. Furthermore, itis possible to generate a pan-sharpened image obtained by compositing amonochrome image and a color image, or a captured image in which theresolution has been increased obtained by compositing captured imageswith different imaging conditions such as different dynamic ranges orshutter speeds, different bands (wavelength bands), or differentresolutions, for example.

(2) Function Assignment

An index such as a normalized difference vegetation index (NDVI) can becalculated by different bands such as red (R) and infrared (IR).

(3) Three-Dimensional Measurement

Three-dimensional information can be obtained from a parallax image.Furthermore, the accuracy of recognizing an object on the ground can beenhanced by the three-dimensional information. For example, it ispossible to determine whether or not the object is a vehicle (even in acase where it is not possible to immediately recognize the object as avehicle from the image in terms of resolving power, if it is determinedthat the object on the road is not a pattern but a three-dimensionalobject, it is possible to estimate that the object is a vehicle).

(4) Differential Measurement

By using a plurality of captured images captured from the same positionwith a time difference, it is possible to extract a change between afirst time and a second time. Furthermore, imaging may be performed suchthat only a target that has changed is extracted and colored.Furthermore, for example, a moving speed of a ship or a vehicle can becalculated from a plurality of captured images, or a wind speed can becalculated from a movement of cloud or the like.

(5) Other Image Compositions

It is also possible to perform composition using a past captured imageor a captured image captured by another satellite 21, composition usingcaptured images captured in different bands, composition using mapinformation, and the like.

The satellite cluster management device 11 and the image analysis server42 as image processing apparatuses perform the above-described imageprocessing on the basis of satellite specification information forspecifying a satellite associated as metadata with a captured imagecaptured by the satellite 21. In other words, since the satellitespecification information is associated as metadata with the capturedimage, it is possible to process a plurality of images by using arelative positional relationship among the plurality of satellites 21 ina formation flight. The satellite specification information includes atleast a cluster ID for identifying the satellite cluster 31, anindividual ID for identifying each satellite 21 constituting thesatellite cluster 31, and information regarding the relative position ofeach satellite 21 that performs the formation flight.

Note that, although image processing using a plurality of capturedimages captured by a formation flight has been described, theabove-described image processing may be performed on a plurality ofcaptured images captured by a constellation instead of a formationflight. For example, image processing such as (1) higher resolution orhigh quality processing, (3) three-dimensional measurement, or (5) otherimage compositions may be performed on a plurality of captured imagescaptured by a constellation.

(Image Format)

Processed images after image processing and captured images are storedin a storage unit and provided to a customer or the like by using, forexample, the following image formats.

(1) CEOS

CEOS is a format standardized by the Committee on Earth ObservationSatellites. CEOS includes “CEOS-BSQ” in which a file is divided for eachband and “CEOS-BIL” in which a plurality of bands is multiplexed.

(2) HDF

This is a format developed by the National Center for SupercomputingApplications (NCSA) at the University of Illinois. A plurality of bandsis grouped into one file so that data can be easily exchanged in a widevariety of computer environments.

(3) Geo TIFF

This is a format in which information for remote sensing is added to atagged image file format (TIFF). This is in the TIFF format, and can beopened with a general image viewer or the like.

(4) JPEG2000

This is an image format standardized by Joint Photographic ExpertsGroup. JPEG 2000 not only simply increases a compression rate, but alsoadopts a technology for improving an image in a region of interest and acopyright protection technology such as an electronic watermark.

Methods for presenting processed images and captured images include (1)a method of providing an image in a manner such that the image can beviewed and (2) a method of presenting only information based on analysisof the image.

Moreover, examples of (1) the method of providing an image in a mannersuch that the image can be viewed include (1A) a method of providing(transmitting) the image itself, (1B) a method of allowing access to aplatform such as a data server and allowing a user to view an image ofdata on the platform, and (1C) a method of providing dedicated softwarefor viewing images to a user and allowing the user to view the imagesonly with the dedicated software.

(2) The method of presenting only information based on analysis of theimage is, for example, a method of presenting the number of vehicles ormoving objects in each time zone or presenting an area of a group offish obtained by the processing of information extraction describedabove.

6. Details of Metadata

FIG. 10 illustrates an example of information attached as metadata to acaptured image or a processed image.

The information attached as metadata includes, depending on the type ofinformation, information that can be added by a satellite 21,information that can be added by the satellite cluster management device11, and information that can be added by the image analysis server 42 ofthe analysis company. In FIG. 10, the types of information are disposedin a table format, and a circle (∘) is attached to a device that can addthe corresponding type of information. Note that, in a case where thesatellite cluster management device 11 also has an image processingfunction, it goes without saying that information that can be added bythe image analysis server 42 can also be added by the satellite clustermanagement device 11 itself.

As the metadata, for example, information for specifying a satellite(satellite specification information) can be added. The information forspecifying a satellite may include, for example, a cluster ID foridentifying the satellite cluster 31, an individual ID for identifyingeach satellite 21, information regarding the relative position of eachsatellite 21 constituting the satellite cluster 31 that performs aformation flight, angle information of itself (the satellite 21) at thetime of imaging, and a satellite type. The information regarding therelative position includes, for example, information such as the orderof the plurality of satellites 21 constituting the satellite cluster 31and the distances between the satellites. The information regarding therelative position may be information that can be used for estimation ofthe relative position. The angle information of itself at the time ofimaging indicates, for example, the angle of itself with respect to theground surface at the time of imaging. The satellite type includes, forexample, whether the satellite is an optical satellite or a SARsatellite, or a division by classification based on usage and size ofthe satellite.

Furthermore, the information for specifying a satellite may include, forexample, orbit information (TLE information) in the TLE format of thesatellite 21, position information (GPS information) by a GPS signal,orbit position/orbit altitude information calculated from at least oneof the TLE information or the GPS information, speed information of thesatellite 21, and sensor information of an earth sensor, a sun sensor, astar tracker, or the like of the satellite 21.

Furthermore, information regarding imaging contents can be added to themetadata. The information regarding imaging contents may include, forexample, imaging target position information indicating a place on theearth as an imaging target, imaging conditions such as the resolution(resolving power), zoom, shutter speed, sensitivity, and aperture(f-number), a sensor type such as the model number of an image sensor,the imaging time, the satellite position at the time of imaging, andweather information such as the cloud cover and amount of sunlight.

As the imaging target position information, for example, informationregarding latitude and longitude of a place on the earth as an imagingtarget is given. The satellite position at the time of imaging is addedon the ground side on the basis of orbit information of the satellite21. The satellite position at the time of imaging may be the orbitinformation of the satellite 21 itself. Furthermore, in the imagingpreparation processing described above, since there is a case where theaccuracy of the attitude control is changed in accordance with theremaining battery level, the satellite position at the time of imagingmay further include information regarding the accuracy of the attitudecontrol for the satellite 21 at the time of imaging, three-dimensionalacceleration information indicating a movement of the satellite itselfat the time of imaging, and the like. The information regarding theattitude control can be used as a reference for processing in, forexample, high resolution processing on a captured image performed on theground side.

Moreover, information regarding an image type can be added to themetadata. The information regarding the image type may include bandinformation and image processing information.

The band information includes wavelength information related to awavelength band, color information indicating RGB (true color), IR(infrared light), or monochrome, coloring information indicating that aspecific target such as a plant has been colored (false color), andanalysis information indicating that the image indicates a normalizeddifference vegetation index (NDVI) or a normalized difference waterindex (NDWI).

The image processing information includes a processing time, aprocessing level, a processing method, and the like of the imageprocessing. The processing time indicates the time when the imageprocessing has been performed. The processing level is divided into sixlevels from L0 to L5. L0 is a level indicating an uncorrected statewhere correction processing has not been performed, L1 is a level wherea radiometric correction related to the sensitivity characteristic hasbeen performed, and L2 is a level where a geometric correction for theorbit position, attitude error, or the like of the satellite 21 has beenperformed. In addition, there are a level where an image has beenprojected on a map projection surface, a level where an ortho-correctionfor correcting geometric distortion has been performed, and the like.Processing methods are described by processing names such aspan-sharpening processing, true color composition processing, and SARimage composition processing. A processed image of the three-dimensionalmeasurement may include a description of distinction between an L image(image for a left eye) and an R image (image for a right eye).

Moreover, related person information, which is information regarding aperson related to a captured image or a processed image, can be added tothe metadata. The information regarding the related person includes, forexample, information regarding an owner of the satellite 21, a serviceoperator who operates a satellite remote sensing service, or a personwho has a right to the captured image or the processed image. By addingthe related person information as metadata to the captured image or theprocessed image, it is possible to manage the person related to thecaptured image or the processed image by referring to or collating theperson related to the captured image or the processed image, andauthenticity of the image can be secured.

7. Details of Distribution Management Processing

Next, the distribution management processing on a captured image or aprocessed image executed by the satellite cluster management device 11or the image analysis server 42 in step S19 in FIG. 6 and step S108 inFIG. 9 will be described.

Captured images and processed images can be subjected to the followingprocessing for managing distribution of data.

(1) Use Limitation Processing

It is possible to perform processing such that captured images andprocessed images cannot be downloaded or displayed without permission,or perform processing such that captured images and processed imagescannot be downloaded or displayed in a case where a predeterminedcondition such as an expiration period, the number of times of copying,or the number of times of displays is satisfied. Furthermore, it ispossible to perform the processing such that secondary processing suchas image composition cannot be performed on captured images andprocessed images.

(2) Watermark

Processing of putting a watermark (electronic watermark) indicatingcopyright can be performed on captured images and processed images.Furthermore, it is possible to perform processing of putting, as awatermark, information that enables determination of a route of leakage.

By performing the distribution management processing as described above,it is possible to secure authenticity of images, and prevent leakage andinappropriate use of captured images and processed images. At this time,a method of using a blockchain to manage each piece of data and a modeof use of the data may be adopted.

(Processing Example of Image Protection)

In a case where a user has requested for privacy protection of capturedimages and processed images, or in a case of images including an area(disclosure-restricted area) disclosure of which is restricted or anarea (prohibited area) disclosure of which is prohibited by a law or thelike of each country, such as a military facility or a public facility,the satellite cluster management device 11 or the image analysis server42 can perform processing of protecting the images by a predeterminedprotection method. Whether or not the area is a protected area may bedetermined with the use of imaging target position information ofmetadata.

Examples of a method of protecting an image include performingprocessing on an image of a protected area such that persons other thanend users and permitted users cannot perform processing for increasingthe resolution more than necessary. Alternatively, an image of aprotected area may be decreased in resolution or blurred. Furthermore,updating of an image of a protected area may be stopped, and the imagemay be replaced with a past image and displayed, or an image indicatingprotection may be superimposed.

As for image protection, in addition to a case where the imageprotection is executed in advance before an image is first provided to auser, the processing can be performed later in a case where a privacyprotection request has been made, in a case where distribution of anillegal image has been detected, or the like. In a case wheredistribution of an illegal image has been detected, it is possible totake measures to delete captured images and processed images that havebeen illegally leaked.

Allowing the satellite cluster management device 11 and the imageanalysis server 42 to perform the image protection processing asdescribed above makes it possible to respond to a user's request forprivacy protection and disclosure restriction.

8. Application Example of Formation Flight

Hereinafter, an example of image analysis processing using capturedimages captured by the plurality of satellites 21 constituting thesatellite cluster 31 by a formation flight will be described.

(1) Checking Germination of Crops by Higher Resolution (Remote Sensingfor Agriculture)

Observation for checking germination of crops requires a resolution ofseveral centimeters. Compositing images captured by a plurality ofsatellites by a formation flight allows for achieving a resolving powerexceeding a resolving power achieved by a single device, and this allowsfor detection of germination.

The satellite cluster 31 captures images with the same point in farmlandas an imaging target position. The satellites 21 may simultaneouslycapture images from different positions, or may capture images from thesame position with a time difference. In order to turn the imagingtarget position of each satellite 21 toward the same point, it isnecessary to grasp the satellite position in advance.

In image composition processing, it is not required to grasp, for eachcaptured image, which satellite 21 has captured the image. However,grasping which satellite 21 has captured the image makes it possible todetermine the angle at the time of imaging and the time, and imagecomposition can be performed more efficiently.

For example, a Geo TIFF format can be used as a format of a processedimage after composition, and information that the processed image is acomposite image by a formation flight, and the imaging position, theimaging time, the imaging conditions, and the like of each capturedimage used for the composition can be attached as metadata. As theinformation regarding the imaging position, information regarding theimaging position of any of the captured images (a representativecaptured image) used for the composition can be used.

(2) Checking Growth Situation of Crops by Three-Dimensional Measurement(Remote Sensing for Agriculture)

A growth situation of crops is checked on the basis of an index such asthe NDVI, or can also be checked by accurately acquiring heightinformation by three-dimensional measurement.

The satellites 21 of the satellite cluster 31 simultaneously captureimages with the same point, which is farmland, as an imaging targetposition, and obtain a parallax image. In order to obtain the distancesbetween the satellites, which is the baseline length, informationregarding the relative position of the satellites 21 is required. Thisinformation regarding the relative position may not be obtained inadvance, but may be obtained simultaneously with the downlink of thecaptured images.

In image composition processing, it is not required to grasp, for eachcaptured image, which satellite 21 has captured the image. However,grasping which satellite 21 has captured the image makes it possible todetermine the angle at the time of imaging and the time, and imagecomposition can be performed more efficiently.

For a processed image after the composition, for example, a format of athree-dimensional image constituted by a set of an L image and an Rimage can be used, and information that the processed image is acomposite image by a formation flight, and the imaging position, theimaging time, the imaging conditions, and the like of each capturedimage used for the composition can be attached as metadata. As theinformation regarding the imaging position, information regarding theimaging position of any of the captured images (a representativecaptured image) used for the composition can be used. In addition toinformation regarding the three-dimensional measurement, a vegetationindex such as the NDVI or another piece of information may be furtheradded.

(3) Other Types of Remote Sensing for Agriculture

For example, it is possible to accurately acquire height information forlevelness check after tilling of farmland by three-dimensionalmeasurement.

4) Detection of Movement of Group of Fish (Ocean Observation RemoteSensing)

A group of fish can be detected, and information regarding a movingdirection and a moving speed of the group of fish can be obtained.

The satellite cluster 31 captures images with the same point in theocean as an imaging target position. The satellites 21 capture imagesfrom the same position with a time difference. In order to turn theimaging target position of each satellite 21 toward the same point, itis necessary to grasp the satellite position in advance. Particularly ina case of imaging in which the ocean where there is no target thatserves as a reference is set as an imaging target position, it isnecessary to precisely align images captured by the satellites 21, andthus, it is important to grasp in advance information regarding therelative position and the moving speed of the satellites 21.

In processing of analyzing the captured images, alignment of the imagescaptured by the satellites 21 and group of fish comparison processingare performed on the basis of the imaging positions (including angleinformation) and the imaging times. By performing the comparisonprocessing, it is possible to calculate the moving speed of the group offish from a time difference between the imaging times of the two or moresatellites 21 and the moving distance of the group of fish.

As an image to be presented as an analyzed image, for example, an imagecan be adopted in which information indicating the moving direction andthe moving speed of the group of fish is superimposed and displayed on acaptured image of the group of fish serving as a base (an image capturedby a predetermined satellite 21). Various types of information of thecaptured image serving as a base are added to the metadata.

As a result of the analysis processing, information describing acalculation method used to calculate the moving direction and the movingspeed of the group of fish may be presented. Examples of thisinformation include a plurality of captured images showing the group offish, and information such as the imaging times of the captured imagesand the position of the group of fish.

(5) Other Types of Ocean Observation Remote Sensing

For example, it is also possible to obtain information regarding themoving direction and the moving speed of a ship and ocean currentobservation information.

(6) Counting the Number of Vehicles (Estimation of Economic Index)

An economic index (business trends or sales prediction of a specificstore) is calculated by examining the number of vehicles in a parkinglot and the number of vehicles running on a road. It is possible togenerate a high-resolution captured image by compositing images capturedby a plurality of satellites by a formation flight and more accuratelydetect the number of vehicles or the number of running vehicles.

The satellite cluster 31 simultaneously captures images with the samepoint as an imaging target position. In order to turn the imaging targetposition of each satellite 21 toward the same point, it is necessary tograsp the satellite position in advance. By using a plurality ofcaptured images that have been captured simultaneously, it is possibleto increase the resolution of an image and acquire three-dimensionalinformation based on a parallax image.

In image composition processing, it is not required to grasp, for eachcaptured image, which satellite 21 has captured the image. However,grasping which satellite 21 has captured the image makes it possible todetermine the angle at the time of imaging and the time, and imagecomposition can be performed more efficiently. In a case of compositionfrom two or more captured images, a target object that serves as areference may be extracted from a road or a building in the images, andthe two or more images may be aligned on the basis of the extractedtarget object. The target object that serves as a reference may beselected on the basis of height information.

In image analysis processing, the number of vehicles or the number ofrunning vehicles is calculated on the basis of a captured image in whichthe resolution has been increased. The number of vehicles or the numberof running vehicles may be efficiently calculated by increasing theresolution only in a specific region in the captured image. In a casewhere it is not possible to determine whether or not a target object isa vehicle from a two-dimensional image, the determination of whether ornot the target object is a vehicle may be made on the basis ofthree-dimensional information including the height.

As an image to be presented as an analyzed image, for example, it ispossible to adopt an image in which a captured image serving as a base(an image captured by a predetermined satellite 21) is colored indifferent colors for each detection target area or each count target(vehicles or persons), and the number of counts is superimposed anddisplayed. Various types of information of the image serving as a baseare given to the metadata.

As a result of the analysis processing, information such as an imagingcondition of the image or a calculation method for the object to bedetected may be presented to a user.

Note that the above-described example is an example of increasing theresolution by simultaneous imaging, and it is also possible to measurethe moving speed of a vehicle on the basis of captured images capturedwith a time difference, and estimate and present traffic volumeinformation before and after the imaging time.

(7) Others

By compositing images captured by a plurality of satellites by aformation flight, it is possible to acquire three-dimensionalinformation based on a parallax image, and create a three-dimensionalmap of a construction site or a house.

(8) Modified Example

A constellation of a formation flight may be used. That is, by puttingthe satellite cluster 31 that performs a formation flight into a singleor a plurality of orbital planes, it is possible to perform an operationof mainly deploying a uniform global service.

Image composition of an image captured by the formation flight and animage captured by another satellite may be performed. For example, it ispossible to perform image processing in which moving object informationobtained by the formation flight is superimposed and displayed on ahigh-resolution image captured by a geostationary satellite.

9. Second Embodiment of Satellite Image Processing System

FIG. 11 illustrates a configuration example of a second embodiment of asatellite image processing system to which the present technology isapplied.

In the first embodiment described above, the satellite cluster 31 thatperforms a formation flight is configured to perform simultaneousimaging or imaging with a time difference at an imaging point or animaging time instructed in advance on the basis of orbit information orthe like of the satellite 21. Therefore, for example, it is not possibleto detect a predetermined event that has occurred on the ground andperform real-time imaging at the time of occurrence of the event.

In the second embodiment described below, a configuration in which oneor more satellites 21 perform real-time imaging in accordance with anevent that has occurred on the ground will be described. In a case wherea satellite cluster 31 including a plurality of the satellites 21performs real-time imaging in accordance with an event that has occurredon the ground, the satellite cluster 31 may be operated by either aconstellation or a formation flight.

In the configuration of a satellite image processing system 1 of thesecond embodiment, as illustrated in FIG. 11, a plurality oftransmission devices 251 including a sensor that detects a predeterminedevent on the ground is newly added. In the example in FIG. 11, fourtransmission devices 251A to 251D are installed in an event detectionregion 250, but the number of transmission devices 251 is optional. Notethat three satellites 21X to 21Z of the second embodiment illustrated inFIG. 11 may be operated by either a constellation or a formation flight.Furthermore, the three satellites 21X to 21Z may be independentlyoperated satellites 21.

The event detection region 250 is divided and assigned to each of thefour transmission devices 251A to 251D for event detection. A fan-shapedregion indicated by a broken line in FIG. 11 indicates an eventdetection range of one transmission device 251. The event detectionregion 250 is, for example, farmland, and the sensors included in thetransmission devices 251 monitor the temperature and the like of thefarmland, or monitor a growth situation of crops.

The transmission devices 251 detect a predetermined event in the eventdetection region 250, and transmit an imaging instruction to one or moresatellites 21. The satellites 21X to 21Z image an occurrence region ofthe event in accordance with the imaging instruction transmitted fromthe transmission devices 251.

FIG. 12 is a block diagram illustrating a configuration example of thetransmission device 251.

The transmission device 251 includes a transmission unit 271, a controlunit 272, a sensor unit 273, and a power supply unit 274.

In accordance with the control of the control unit 272, the transmissionunit 271 transmits an imaging instruction to a satellite 21 passingthrough the vicinity of the transmission device 251.

The transmission unit 271 is, for example, omnidirectional, and cantransmit an imaging instruction to all the satellites 21 passing througha certain range of the transmission device 251. The transmission unit271 is constituted by, for example, a communication device that cancommunicate with an object moving at a high speed of 100 km/h over along distance of 100 km or more, and consumes less power.

The transmission unit 271 may be directional. In this case, thetransmission unit 271 directs an antenna (not illustrated) toward asatellite 21 passing through the vicinity of the transmission device 251on the basis of orbit information of the satellite 21, and transmits animaging instruction to the target satellite 21. The orbit information ofthe satellite 21 is stored in advance.

The control unit 272 controls the entire operation of the transmissiondevice 251. In a case where a predetermined event has been detected bythe sensor unit 273, the control unit 272 performs control to cause thetransmission unit 271 to transmit an imaging instruction to thesatellite 21.

The sensor unit 273 is constituted by one or more types of predeterminedsensors in accordance with the purpose of event detection. For example,the sensor unit 273 is constituted by an odor sensor, an atmosphericpressure sensor, a temperature sensor, and the like. Furthermore, forexample, the sensor unit 273 may be constituted by an image sensor (RGBsensor, IR sensor, or the like) that images the event detection region250. For example, when a detection value is equal to or more than apredetermined threshold, the sensor unit 273 detects occurrence of anevent, and notifies the control unit 272 of the occurrence of the event.

Note that the sensor unit 273 may be disposed close to the transmissionunit 271, or may be disposed away from the transmission unit 271 in sucha way that, for example, the transmission unit 271 is disposed at a highplace closest to the satellite 21, and the sensor unit 273 is disposedat a low place close to the ground.

A plurality of sensors of different types may be mounted on onetransmission device 251, or a plurality of sensors of the same type maybe mounted. In a case where a plurality of sensors is mounted on thetransmission device 251, there is a case where it is necessary totransmit a sensor detection result with sensor information such as asensor detection range as an imaging target position or a sensordetection type as transmission information added.

The power supply unit 274 is constituted by, for example, a batterycharged by solar power generation or the like, and supplies power toeach unit of the transmission device 251.

The transmission device 251 is a communication device that has aconfiguration as described above and allows for only unidirectionalcommunication from the transmission device 251 to the satellite 21, butmay also be a communication device that allows for bidirectionalcommunication including a direction from the satellite 21 to thetransmission device 251.

In both the one-way communication and the bidirectional communication,in a case where the communication is omnidirectional, it is notnecessary for a transmission side to direct an antenna toward thesatellite 21 or a ground station 15, which is a reception side, and thussuch communication is preferable particularly in a case of transmissionfrom the ground to the satellite 21 overhead. In the present embodiment,it is assumed that the transmission unit 271 of the transmission device251 is omnidirectional and the transmission device 251 is a device thatperforms one-way communication. However, as a matter of course, thetransmission device 251 may be a directional device that performsbidirectional communication.

10. First Event Imaging Sequence of Second Embodiment

Next, a first event imaging sequence performed by the satellite imageprocessing system 1 of the second embodiment will be described withreference to a flowchart in FIG. 13.

First, in step S141, the control unit 272 of the transmission device 251determines whether an event has been detected by the sensor unit 273.When the sensor unit 273 detects a predetermined event and notifies thecontrol unit 272 of occurrence of the event, the control unit 272determines that an event has been detected. Thus, in step S141, thecontrol unit 272 waits until a notification of occurrence of an event isreceived from the sensor unit 273. If it is determined that an event hasbeen detected, the processing proceeds from step S141 to step S142.

In response to the occurrence of the event, in step S142, the controlunit 272 controls the transmission unit 271 to transmit an imaginginstruction to a satellite 21 passing through the vicinity of thetransmission device 251. The transmission unit 271 transmits the imaginginstruction in response to a command from the control unit 272.

Since the communication between the transmission device 251 and thesatellite 21 is one-way communication only from the ground side to thesatellite 21, the transmission device 251 cannot check whether or notthe satellite 21 has received the imaging instruction. Therefore, thetransmission device 251 continues to transmit the imaging instructionfor a certain period of time such as thirty minutes or one hour, orrepeatedly transmits the imaging instruction intermittently at a certaintime interval. In a case where the transmission device 251 and thesatellite 21 can perform bidirectional communication, as in the imagingsequence described with reference to FIG. 6, a reception completion maybe received from the satellite 21, and then the transmission of theimaging instruction may be stopped. The reception completion from thesatellite 21 to the transmission device 251 may include information thatthe satellite 21 will capture an image.

Furthermore, in the present imaging sequence, when occurrence of anevent has been detected, the transmission device 251 transmits animaging instruction without selecting a satellite 21. Alternatively, ina case where orbit information and an imaging capability of a satellite21 passing overhead are known, the transmission device 251 may transmitan imaging instruction in which the satellite cluster 31 or thesatellite 21 that satisfies requested imaging conditions is designatedby the cluster ID or the individual ID.

The imaging instruction from the transmission device 251 to thesatellite 21 is transmitted with imaging-related information such asrequested imaging conditions, a requested imaging target position, asensor ID, an event occurrence time, and a detected event type added asparameters. The requested imaging conditions include, for example,resolution and a wavelength band (RGB, IR, or the like). The requestedimaging target position represents a region on the ground to be imaged,and corresponds to an occurrence region of the event of the sensor unit273. An installation position of the transmission device 251 or thesensor unit 273 may be stored as the requested imaging target position.The sensor ID is sensor identification information for identifying thesensor unit 273 that has detected the event. The event occurrence timeis a time at which the sensor unit 273 has detected the event, andcorresponds to a time at which a request has been made as the imaginginstruction. The detected event type indicates, for example, the type ofevent detected by the sensor unit 273, such as detection of an abnormaltemperature. The detected event type may store the sensor type insteadof a specific type of the detected event.

In step S161, the satellite 21 receives the imaging instruction from thetransmission device 251, and in step S162, determines whether imaging byitself is possible. The satellite 21 checks whether or not the requestedimaging conditions added to the imaging instruction are satisfied, anddetermines whether imaging by itself is possible. If it is determined instep S162 that imaging by itself is not possible, the satellite 21 endsthe processing.

On the other hand, if it is determined in step S162 that imaging byitself is possible, the processing proceeds to step S163, and thesatellite 21 performs imaging preparation processing based on thereceived imaging instruction. Subsequently, the satellite 21 captures animage in step S164, and generates metadata and adds the metadata to thecaptured image in step S165. Since each piece of processing in stepsS163 to S165 is basically similar to each piece of processing in stepsS33 to S35 in FIG. 6 described above, the details thereof will beomitted. The metadata can include a part or all of the informationreceived from the transmission device 251. For example, information suchas the sensor ID indicating the sensor unit 273 or the event occurrencetime can be included as the metadata.

In step S166, the satellite 21 determines whether the satellite 21 hasarrived at a downlink point, in other words, whether the satellite 21has arrived within a range in which communicate with a communicationdevice 13 of the ground station 15 is possible. The satellite 21 repeatsthe processing in step S166 until it is determined that the satellite 21has arrived at the downlink point. If it is determined that thesatellite 21 has arrived at the downlink point, the processing proceedsto step S167.

In step S167, the satellite 21 transmits (downlinks) the captured imageto which the metadata has been added to the ground station 15. Thedownlink may be performed via a relay satellite 22.

In step S181, a management system receives the captured image from thesatellite 21. That is, the communication device 13 receives the capturedimage via an antenna 14, and supplies the captured image to a satellitecluster management device 11. After receiving the captured image, themanagement system performs processing similar to that in steps S17 toS19 in FIG. 6, and the description thereof will not be repeated.

11. Second Event Imaging Sequence of Second Embodiment

Next, a second event imaging sequence performed by the satellite imageprocessing system 1 of the second embodiment will be described withreference to a flowchart in FIG. 14.

In the first event imaging sequence described above, each satellite 21individually determines whether or not imaging is possible, andtransmits a captured image to the communication device 13 on the groundin a case where imaging is performed.

In the second event imaging sequence in FIG. 14, processing has beenadded in which, in a case where a satellite 21 that has received animaging instruction determines that imaging by itself is not possible, asubsequent satellite 21 takes over the imaging instruction. Thesubsequent satellite 21 is, for example, a satellite 21 belonging to thesame satellite cluster 31 operated in a constellation or a formationflight. In the second event imaging sequence described below, thesatellite 21 that receives the imaging instruction is referred to as thefirst satellite 21, and the subsequent satellite 21 that takes over theimaging instruction is referred to as the second satellite 21 fordistinction.

Detection of occurrence of an event in steps S141 and S142 andtransmission of an imaging instruction by the transmission device 251are the same as those in the first event imaging sequence describedabove.

In step S201, the first satellite 21 receives the imaging instructionfrom the transmission device 251, and in step S202, determines whetherimaging by itself is possible. if it is determined in step S202 thatimaging by itself is possible, the processing proceeds to step S203, andthe first satellite 21 performs imaging based on the imaging instructionand transmission, and the processing ends. The imaging sequence in acase where it is determined that imaging by itself is possible is thesame as that in the first event imaging sequence described above, andthus, description thereof will be omitted.

On the other hand, if it is determined in step S202 that imaging byitself is not possible, the processing proceeds to step S204, and thefirst satellite 21 determines whether imaging by the subsequent secondsatellite 21 belonging to the satellite cluster 31 of the firstsatellite 21 is possible. If it is determined in step S204 that imagingby the second satellite 21 is not possible, the processing ends.

If it is determined in step S204 that imaging by the second satellite 21is possible, the processing proceeds to step S205, and the firstsatellite 21 transmits the imaging instruction to the subsequent secondsatellite 21 by inter-satellite communication.

Then, in step S206, the first satellite 21 determines whether the firstsatellite 21 has arrived at the downlink point, and repeats theprocessing in step S206 until it is determined that the first satellite21 has arrived at the downlink point.

Then, if it is determined in step S206 that the first satellite 21 hasarrived at the downlink point, the processing proceeds to step S207, andthe first satellite 21 transmits (downlinks), to the ground station 15,event detection data included in the imaging instruction received fromthe transmission device 251. The event detection data includes a part orall of imaging-related information included in the imaging instruction,information that the imaging instruction has been transferred to thesubsequent satellite, and information indicating the subsequent secondsatellite 21 to which the imaging instruction has been transferred. Thedownlink may be performed via the relay satellite 22 in a similar mannerto another piece of processing described above. Thus, the processing bythe first satellite 21 ends.

The subsequent second satellite 21 to which the imaging instruction hasbeen transmitted from the first satellite 21 by inter-satellitecommunication receives the imaging instruction in step S221, andperforms imaging preparation processing based on the received imaginginstruction in step S222.

The processing in steps S223 to S226 is similar to the processing insteps S164 to S167 in FIG. 13. By the processing in steps S223 to S226,imaging is performed, a captured image and metadata are generated, andthe captured image to which the metadata has been added is transmittedto the ground station 15 at the time of arrival at the downlink point.

On the other hand, in response to the transmission of the eventdetection data by the first satellite 21, the management system receivesthe event detection data in step S241. Furthermore, in response to thetransmission of the captured image by the second satellite 21, thecaptured image is received in step S242. After receiving the capturedimage, the management system performs processing similar to that insteps S17 to S19 in FIG. 6, and the description thereof will not berepeated.

12. Third Event Imaging Sequence of Second Embodiment

Next, a third event imaging sequence performed by the satellite imageprocessing system 1 of the second embodiment will be described withreference to a flowchart in FIG. 15.

In the second event imaging sequence described above, inter-satellitecommunication is used for transfer of an imaging instruction from thefirst satellite 21 to the second satellite 21. The third event imagingsequence is an example in which communication via the ground station 15is used for transfer of an imaging instruction from the first satellite21 to the second satellite 21.

Detection of occurrence of an event in steps S141 and S142 andtransmission of an imaging instruction by the transmission device 251are the same as those in the first event imaging sequence describedabove.

In step S301, the first satellite 21 receives the imaging instructionfrom the transmission device 251, and in step S302, determines whetherimaging by itself is possible. If it is determined in step S302 thatimaging by itself is possible, the processing proceeds to step S303, andthe first satellite 21 performs imaging based on the imaging instructionand transmission, and the processing ends. The imaging sequence in acase where it is determined that imaging by itself is possible is thesame as that in the first event imaging sequence described above, andthus, description thereof will be omitted.

On the other hand, if it is determined in step S302 that imaging byitself is not possible, the processing proceeds to step S304, and thefirst satellite 21 determines whether imaging by the subsequentsatellite 21 belonging to the satellite cluster 31 of the firstsatellite 21 is possible. If it is determined in step S304 that imagingby the subsequent satellite 21 is possible, the processing proceeds tostep S305, imaging and transmission by the subsequent satellite 21 areperformed, and the processing ends. An imaging sequence in a case whereit is determined that imaging by the subsequent satellite 21 is possibleis the same as that in the above-described second event imagingsequence, and thus, description thereof will be omitted.

If it is determined in step S304 that imaging by the subsequentsatellite 21 is not possible, the processing proceeds to step S306, andthe first satellite 21 determines whether the first satellite 21 hasarrived at the downlink point, and repeats the processing in step S306until it is determined that the first satellite 21 has arrived at thedownlink point.

Then, if it is determined in step S306 that the first satellite 21 hasarrived at the downlink point, the processing proceeds to step S307, andthe first satellite 21 transmits (downlinks), to the ground station 15,the imaging instruction received from the transmission device 251. Thedownlink may be performed via the relay satellite 22 in a similar mannerto another piece of processing described above. Thus, the processing bythe first satellite 21 ends.

In response to the transmission of the imaging instruction by the firstsatellite 21, the management system receives the imaging instruction instep S321. Then, in step S322, the management system specifies anothersatellite 21 that satisfies requirements for imaging on the basis of therequested imaging conditions, the requested imaging target position, andthe like included in the imaging-related information of the imaginginstruction. Here, the second satellite 21 is specified as the othersatellite 21.

In step S323, the management system transmits the imaging instruction tothe specified second satellite 21. Note that the ground station 15 (thecommunication device 13 thereof) that receives the imaging instructionfrom the first satellite 21 and the ground station 15 (the communicationdevice 13 thereof) that transmits the imaging instruction to the secondsatellite 21 may be the same, or may be different.

In step S341, the second satellite 21 receives the imaging instructionfrom the ground station 15. The processing in the following steps S342to S346 is similar to the processing in steps S222 to S226 in FIG. 14,and thus, description thereof will be omitted. In step S346, thecaptured image is transmitted from the second satellite 21 to themanagement system.

In step S324, the management system receives the captured image, and thethird event imaging sequence ends.

In the third event imaging sequence described above, the first satellite21 transmits the imaging instruction to the ground station 15 if it isdetermined that imaging by the subsequent satellite 21 is not possible.Alternatively, the first satellite 21 may transmit the imaginginstruction to the ground station 15 if it is determined that imaging byitself is not possible, without determining whether or not imaging bythe subsequent satellite 21 is possible.

According to the third event imaging sequence, even in a case where therequested imaging target position is a place where connection to anetwork is not available, such as on the sea, an imaging instruction canbe transmitted to the management system via the first satellite 21, andimaging can be performed by the second satellite 21.

13. Another Configuration Example of Transmission Device

The transmission device 251 illustrated in FIG. 12 has the built-insensor that detects occurrence of an event, and is configured integrallywith the transmission unit that transmits an imaging instruction.However, the sensor that detects occurrence of an event and thetransmission device that transmits an imaging instruction can beconstituted by separate devices.

FIG. 16 is a block diagram illustrating another configuration example ofthe transmission device according to the second embodiment.

In the event detection region 250 (FIG. 11), a transmission device 291,a control device 292, and one or more sensors 293 are installed. FIG. 16illustrates an example in which the number of the sensors 293 is three,which is constituted by sensors 293A to 293C, but the number of sensors293 is optional. Furthermore, a plurality of sets of the transmissiondevice 291, the control device 292, and one or more sensors 293 may beinstalled in the event detection region 250.

In accordance with the control of the control device 292, thetransmission device 291 transmits an imaging instruction to a satellite21 passing through the vicinity of the transmission device 291.

In a case where a predetermined event has been detected by any one of aplurality of sensors 293 (293A to 293C), the control device 292 acquiresan event detection result from the sensor 293, generates an imaginginstruction, and performs control to cause the transmission device 291to transmit the imaging instruction. In a similar manner to theabove-described example, imaging-related information is added to theimaging instruction as parameters.

Each one of the plurality of sensors 293 (293A to 293C) corresponds tothe sensor unit 273 described above, detects occurrence of an event, andnotifies the control device 292 of the occurrence of the event. Theplurality of sensors 293 may be constituted by different types ofsensors, or may be sensors of the same type. The plurality of sensors293 may be disposed close to or away from each other. Furthermore, theplurality of sensors 293 may be disposed close to or away from thetransmission device 291 and the control device 292. The above-describedsensor information is added to a notification of occurrence of an eventfrom the sensors 293 to the control device 292 as necessary.

In the satellite image processing system 1 of the second embodiment,even in a case where the transmission device 291 and the sensors 293 areconfigured as separate devices as described above, the first to thirdevent imaging sequences described above can be executed in a similarmanner.

14. Application Examples of Satellite Image Processing System UsingEvent Detection Sensor

Hereinafter, application examples of the satellite image processingsystem using the event detection sensor of the second embodiment will bedescribed.

(1) Event Detection in Farmland

A plurality of sensors (the transmission device 251 including the sensorunit 273, or the sensors 293) is installed at a certain interval in apredetermined observation region in farmland, and each one of theplurality of sensors detects an abnormality such as vermination oroccurrence of a disease. The transmission device 251 or 291 transmits animaging instruction to the satellite 21 in accordance with a result ofdetection of an abnormality in the farmland as an event. The satellite21 performs, for example, imaging of RGB, imaging of red (R) andinfrared (IR) for a vegetation index such as the NDVI, or the like. Thesensor detection range of the sensor that has detected the abnormalityis assigned to the requested imaging target position added to theimaging instruction. The satellite 21 that has received the imaginginstruction may image only the sensor detection range of the sensor inwhich the abnormality has occurred, in the observation region in whichthe plurality of sensors is disposed, or may perform wide-area imagingof the entire observation region. Furthermore, an imaging condition suchas zoom may be changed so that both imaging of the sensor detectionrange of the sensor that has detected the abnormality and wide-areaimaging of the entire observation region may be performed.

It is also possible to give an imaging instruction to the satellite 21by using, as a trigger, occurrence of a predetermined situation forchecking a growing situation, such as the ground surface having got intoa predetermined environmental state (e.g., the temperature of the groundsurface having reached a predetermined temperature), an amount ofphotosynthesis or a growth situation of a plant having got into apredetermined state, or germination having been detected, instead ofdetection of an abnormality.

(2) Event Detection in Ocean

For example, a buoy incorporating the transmission device 251 includingthe sensor unit 273 is released into a sea area to be investigated inthe ocean. The sensor unit 273 detects a group of fish, or detects apredetermined condition such as a sea water temperature, an oceancurrent speed, or a wind speed. On the basis of a result of the eventdetection, the transmission device 251 transmits an imaging instructionto the satellite 21. Imaging-related information of the imaginginstruction includes requested imaging conditions, a requested imagingtarget position, an event occurrence time, and the like. Sincesatellites 21 that can image a state during the night are limited, asatellite 21 is selected on the basis of the requested imagingconditions, and a situation of an imaging target sea area is analyzed onthe basis of a captured image.

(3) Observation of Uninhabited Zone

A sensor (the transmission device 251 including the sensor unit 273, orthe sensor 293) is installed in an uninhabited zone such as a forest, amountain, or a desert, and an abnormality such as a change in climaticcondition, detection of an organism to be observed, or a forest fire isdetected. The satellite 21 captures an image on the basis of an imaginginstruction from the transmission device 251 or 291. On the basis of thecaptured image, the situation of the uninhabited zone is analyzed.

(4) Accident Observation

For example, the transmission device 251 is mounted on a black box of anairplane or a ship, and the transmission device 251 transmits an imaginginstruction in the event of an emergency such as a crash of theairplane, ship grounding, or a leak from an oil tanker. The satellite 21promptly captures an image of the place where the emergency hasoccurred, and transmits the image to the ground station 15.

(5) Stranded Mountain Climber

When a mountain climber or the like carrying the transmission device 251is stranded, the transmission device 251 transmits, to the satellite 21,an imaging instruction to which imaging-related information including adistress signal as a detected event type and including the place wherethe stranding has occurred as a requested imaging target position isadded. The satellite 21 captures an image of the place where thestranding has occurred on the basis of the imaging instruction, andtransmits the image to the ground station 15.

(6) Pipeline Emission Control

Sensors are attached to a pipeline at a predetermined interval, andoccurrence of a leak is monitored. In a case where a leak has beendetected, an imaging instruction is transmitted to the satellite 21. Animaging instruction is transmitted with imaging-related informationdesignating a satellite 21 capable of detecting a leak, such as asatellite 21 capable of detecting heat by an IR band, added as requestedimaging conditions, and a satellite 21 that satisfies requirementscaptures an image. It is possible to promptly observe the situation ofthe leak in the area of the leak on the basis of the captured image. Inparticular, in a case where the leakage from the pipeline ishuman-caused, prompt observation after occurrence of the event iseffective.

(7) Others

A captured image triggered by the sensor 293 disposed on the ground maybe used only as primary information, and the captured image may becombined with another image for image analysis or the like. For example,a captured image triggered by the sensor 293 is promptly captured by alow-performance satellite 21 with priority given to the timing ofimaging. Thereafter, the satellite cluster management device 11 sets aschedule of a satellite 21 having a higher imaging capability forhigh-resolution and high-accuracy imaging. The satellite clustermanagement device 11 performs analysis by using the first captured imagecaptured by the low-performance satellite 21 and the second capturedimage captured by the satellite 21 having a higher imaging capability.For example, the satellite cluster management device 11 may increase theresolution of the first captured image on the basis of differentialinformation, or may perform processing of compositing the first capturedimage and the second captured image.

As described above, according to satellite remote sensing using asensor, an event that has occurred on the ground can be detected by thesensor, and an imaging instruction can be directly given to a satellite21 overhead. In particular, even from a sensor installed in an area thatis not connected to the Internet such as the ocean, an imaginginstruction can be directly given to a satellite, or an imaginginstruction can be given via a satellite to another satellite. Forexample, since it is possible to instantly detect an event that hasoccurred at a specific place in a vast area and cause imaging to beperformed, labor can be greatly reduced.

15. Configuration Example of Computer

The series of pieces of processing described above can be executed notonly by hardware but also by software. In a case where the series ofpieces of processing is executed by software, a program constituting thesoftware is installed on a computer. Here, the computer includes amicrocomputer incorporated in dedicated hardware, or a general-purposepersonal computer capable of executing various functions with variousprograms installed therein, for example.

FIG. 17 is a block diagram illustrating a configuration example ofhardware of a computer that executes the series of pieces of processingdescribed above in accordance with a program.

In the computer, a central processing unit (CPU) 301, a read only memory(ROM) 302, and a random access memory (RAM) 303 are connected to eachother by a bus 304.

The bus 304 is further connected with an input/output interface 305. Theinput/output interface 305 is connected with an input unit 306, anoutput unit 307, a storage unit 308, a communication unit 309, and adrive 310.

The input unit 306 includes a keyboard, a mouse, a microphone, a touchpanel, an input terminal, or the like. The output unit 307 includes adisplay, a speaker, an output terminal, or the like. The storage unit308 includes a hard disk, a RAM disk, a nonvolatile memory, or the like.The communication unit 309 includes a network interface or the like. Thedrive 310 drives a removable recording medium 311 such as a magneticdisk, an optical disk, a magneto-optical disk, or a semiconductormemory.

To perform the series of pieces of processing described above, thecomputer configured as described above causes the CPU 301 to, forexample, load a program stored in the storage unit 308 into the RAM 303via the input/output interface 305 and the bus 304 and then execute theprogram. The RAM 303 also stores, as appropriate, data or the likenecessary for the CPU 301 to execute various types of processing.

The program to be executed by the computer (CPU 301) can be provided by,for example, being recorded on the removable recording medium 311 as apackage medium or the like. Furthermore, the program can be provided viaa wired or wireless transmission medium such as a local area network,the Internet, or digital satellite broadcasting.

Inserting the removable recording medium 311 into the drive 310 allowsthe computer to install the program into the storage unit 308 via theinput/output interface 305. Furthermore, the program can be received bythe communication unit 309 via a wired or wireless transmission mediumand installed into the storage unit 308. In addition, the program can beinstalled in advance in the ROM 302 or the storage unit 308.

In the present specification, the steps described in the flowcharts maybe of course performed in chronological order in the order described, ormay not necessarily be processed in chronological order. The steps maybe executed in parallel, or at a necessary timing such as in a casewhere called.

Furthermore, in the present specification, a system means a set of aplurality of components (devices, modules (parts), and the like), and itdoes not matter whether or not all components are in the same housing.Thus, a plurality of devices housed in separate housings and connectedvia a network, and one device having a plurality of modules housed inone housing are both systems.

Embodiments of the present technology are not limited to the embodimentsdescribed above but can be modified in various ways within a scope ofthe present technology.

For example, it is possible to adopt a mode in which all or some of theplurality of embodiments described above are combined.

For example, the present technology can have a cloud computingconfiguration in which a plurality of devices shares one function andcollaborates in processing via a network.

Furthermore, each step described in the flowcharts described above canbe executed by one device or can be shared by a plurality of devices.

Moreover, in a case where a plurality of pieces of processing isincluded in one step, the plurality of pieces of processing included inthat step can be executed by one device or can be shared by a pluralityof devices.

Note that the effects described in the present specification are merelyexamples and are not restrictive, and effects other than those describedin the present specification may be obtained.

Note that the present technology can be configured as described below.

(1)

An image management method including:

adding,

by a management device that manages a captured image captured by asatellite,

metadata that includes at least information regarding a person relatedto the captured image, to the captured image.

(2)

The image management method according to (1), in which

the related person information includes information regarding any of anowner of the satellite, a service operator using the satellite, or aperson who has a right to the captured image.

(3)

The image management method according to (1) or (2), in which

the management device performs use limitation processing on the capturedimage.

(4)

The image management method according to (1) or (2), in which

the management device performs watermark processing on the capturedimage.

(5)

The image management method according to any one of (1) to (4), in which

the management device receives, from the satellite, the captured imageto which the metadata is added.

(6)

The image management method according to any one of (1) to (4), in which

the management device receives the metadata from the satellite as astream different from the captured image.

(7)

The image management method according to any one of (1) to (6), in which

the metadata includes a satellite identifier for identifying thesatellite and a satellite cluster identifier for identifying a satellitecluster that includes the satellite.

(8)

The image management method according to any one of (1) to (7), in which

the metadata includes information regarding a relative position withrespect to each satellite constituting a satellite cluster.

(9)

The image management method according to (8), in which

the information regarding the relative position includes an arrangementsequence of each satellite constituting the satellite cluster anddistances between the satellites.

(10)

The image management method according to any one of (1) to (9), in which

the metadata includes information regarding imaging contents of imagingby the satellite.

(11)

The image management method according to (10), in which

the information regarding the imaging contents includes at least any oneof imaging target position information, an imaging condition, an imagesensor type, an imaging time, or a satellite position at time ofimaging.

(12)

The image management method according to any one of (1) to (11), inwhich

the metadata includes information regarding an image type of thecaptured image.

(13)

The image management method according to (12), in which

the information regarding the image type of the captured image includesimage processing information.

(14)

A data structure of metadata of a captured image captured by asatellite,

in which the metadata includes at least information regarding a personrelated to the captured image, and

a management device that manages the captured image is used forprocessing of collating the person related to the captured image.

(15)

The data structure of metadata according to (14), in which

the metadata is attached to the captured image.

REFERENCE SIGNS LIST

-   1 Satellite image processing system-   11 Satellite cluster management device-   13 Communication device-   14 Antenna-   15 Ground station (base station)-   21 Satellite-   31 Satellite cluster-   41 Information provision server-   42 Image analysis server-   101 Management unit-   111 Imaging device-   211 Control unit-   222 Control unit-   231 Control unit-   250 Event detection region-   251 Transmission device-   271 Transmission unit-   272 Control unit-   273 Sensor unit-   291 Transmission device-   292 Control device-   293 Sensor-   301 CPU-   302 ROM-   303 RAM-   306 Input unit-   307 Output unit-   308 Storage unit-   309 Communication unit-   310 Drive

1. An image management method comprising: adding, by a management devicethat manages a captured image captured by a satellite, metadata thatincludes at least information regarding a person related to the capturedimage, to the captured image.
 2. The image management method accordingto claim 1, wherein the related person information includes informationregarding any of an owner of the satellite, a service operator using thesatellite, or a person who has a right to the captured image.
 3. Theimage management method according to claim 1, wherein the managementdevice performs use limitation processing on the captured image.
 4. Theimage management method according to claim 1, wherein the managementdevice performs watermark processing on the captured image.
 5. The imagemanagement method according to claim 1, wherein the management devicereceives, from the satellite, the captured image to which the metadatais added.
 6. The image management method according to claim 1, whereinthe management device receives the metadata from the satellite as astream different from the captured image.
 7. The image management methodaccording to claim 1, wherein the metadata includes a satelliteidentifier for identifying the satellite and a satellite clusteridentifier for identifying a satellite cluster that includes thesatellite.
 8. The image management method according to claim 1, whereinthe metadata includes information regarding a relative position withrespect to each satellite constituting a satellite cluster.
 9. The imagemanagement method according to claim 8, wherein the informationregarding the relative position includes an arrangement sequence of eachsatellite constituting the satellite cluster and distances between thesatellites.
 10. The image management method according to claim 1,wherein the metadata includes information regarding imaging contents ofimaging by the satellite.
 11. The image management method according toclaim 10, wherein the information regarding the imaging contentsincludes at least any one of imaging target position information, animaging condition, an image sensor type, an imaging time, or a satelliteposition at time of imaging.
 12. The image management method accordingto claim 1, wherein the metadata includes information regarding an imagetype of the captured image.
 13. The image management method according toclaim 12, wherein the information regarding the image type of thecaptured image includes image processing information.
 14. A datastructure of metadata of a captured image captured by a satellite,wherein the metadata includes at least information regarding a personrelated to the captured image, and a management device that manages thecaptured image is used for processing of collating the person related tothe captured image.
 15. The data structure of metadata according toclaim 14, wherein the metadata is attached to the captured image.